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FOREWORD 


This  report  focuses  on  a  review  of  the  data  bank  developed  with  the  Personnel 
Distribution  and  Career  Development  (PDCD)  work  unit  prior  to  establishing  guidelines  to 
apply  in  the  analysis  of  the  data.  This  initial  report  reflects  general  concerns  pertaining 
to  analytical  strategies  and  establishes  the  parameters  for  identifying  assumptions  and 
specifying  the  analytical  strategies  that  will  be  detailed  in  the  second  report. 

This  is  the  first  of  two  reports  completed  with  TCN  87-621;  the  contracting  officer's 
technical  representative  for  the  work  was  Robert  F.  Morrison.  TCN  87-621  was 
conducted  within  exploratory  development  (Program  Element  0602233N)  under  work  unit 
14SSUX4B529,  PDCD,  sponsored  by  the  Chief  of  Naval  Research  (ONR-222).  This  report 
is  the  14th  published  within  PDCD  and  is  intended  for  use  in  the  PDCD  work  unit. 


JOHN.  3.  PASS 
Director,  Personnel  Systems  Department 
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SUMMARY 


Problem 

A  large  data  bank  has  been  developed  by  the  Personnel  Distribution  and  Career 
Development  (PDCD)  work  unit  for  the  purpose  of  establishing  empirically-based  decision 
guides  to  assist  in  the  design  and  implementation  of  career  policy  and  practice  in  the  U.S. 
Navy.  Data  banks  of  this  magnitude  often  engender  special  methodological  problems 
during  analysis. 

Purpose 

To  identify  salient  methodological  issues  and  problems  likely  to  be  encountered  in 
tests  of  hypothesized  career  models  using  large  banks  of  data  obtained  on  U.S.  Navy 
personnel. 

Approach 

Samples  and  domains  of  measurement  instruments  used  by  the  PDCD  work  unit  in  the 
career  development  research  are  reviewed.  Attention  then  proceeds  to  reviews  of  career 
models  and  the  procedures  that  will  be  employed  to  test  the  hypothesized  career  models. 
Using  these  reviews  as  a  frame  of  reference,  important  methodological  issues  and 
problems  are  identified  and  discussed.  These  issues/problems  include  the  use  of 
confirmatory  analytic  methods  to  analyze  the  data,  procedures  for  testing  for  moderators, 
and  methods  for  increasing  reliability  of  composites  of  items. 
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INTRODUCTION 


In  this  first  of  two  reports,  attention  is  focused  initially  on  a  review  of  the  data  bank 
developed  by  the  Personnel  Distribution  and  Career  Development  (PDCD)  work  unit  for 
the  purpose  of  establishing  empirically-based  decision  guides  to  assist  in  the  design  and 
implementation  of  career  policy  and  practice  in  the  U.S.  Navy.  Samples  and  domains  of 
measurement  instruments  are  reviewed.  Attention  proceeds  to  hypothesized  career 
models  and  the  procedures  that  will  be  used  to  test  the  hypothesized  career  model. 
Salient  methodological  issues  and  problems  likely  to  be  encountered  in  the  tests  of  the 
model  are  considered.  It  is  noteworthy,  however,  that  this  is  an  initial  report  that 
reflects  only  general  and  basic  concerns  pertaining  to  analytic  strategies.  The  second 
report  will  be  devoted  specifically  to  delineating  an  analytic  strategy  and  hence  will 
possess  greater  specificity  and  breadth  in  regard  to  recommended  analyses. 


SAMPLES  AND  INSTRUMENTS 


Samples 

Two  waves  of  questionnaire  data  were  collected  on  samples  of  Navy  officers.  The 
first  wave  was  collected  in  1982  and  is  referred  to  as  the  "1982  wave."  As  described  in 
Morrison  and  Cook  (1985),  the  1982  wave  involved  three  occupational  groups  or  "commun¬ 
ities."  The  communities  are  (1)  Surface  Warfare  Officers  (SWO),  (2)  General  Unrestricted 
Line  (URL(G)),  which  is  primarily  female  (80%)  and  includes  occupational  specialities  such 
as  management  of  shore  installations,  and  (3)  Air  Warfare  Officers  (AWO),  which 
encompasses  the  two  subcommunities  of  pilots  and  Naval  Flight  Officers  (NFO),  the  latter 
being  composed  of  individuals  with  skills  in  electronics  and  navigation.  Sample  sizes  for 
communities  are  2,833  for  SWO,  960  for  URL(G),  and  5,040  for  AWO  (Morrison  <5 c  Cook, 
1985,  Table  1).  The  sampling  design  described  in  Morrison  and  Cook  (1985)  placed  special 
emphasis  on  the  sampling  of  cohorts,  defined  by  year  of  commissioning.  All  officers 
commissioned  in  the  years  1961  through  1980  were  contacted  to  be  sampled  for  the 
URL(G)  community,  thus  providing  20  cohorts  of  individuals  ranging  in  grade  from  Ensign 
to  Captain.  For  the  SWO  and  AWO  communities,  samples  were  drawn  from  officers 
commissioned  in  the  years  1961  through  1976  using  the  same  sampling  plan  as  employed 
for  the  URL(G)  community,  but  officers  commissioned  in  the  years  1977  through  1980 
were  sampled  using  a  different  plan.  Nevertheless,  20  cohorts  are  again  represented  for 
each  (sub)  community,  with  accompanying  variation  in  rank. 

Return  rates  for  the  1982  wave  averaged  approximately  40  percent  over 
communities.  Multiple  reasons  are  apparent  for  this  return  rate,  but  the  need  to  assess 
data  for  bias  using  variables  such  as  demographic  indicators  is  needed.  It  appears  that 
attention  has  already  been  devoted  to  this  issue  (see  Footnote  3  in  Wilcove,  Bruni,  & 
Morrison,  1987). 

The  second  wave  of  data,  the  1986  wave,  is  more  complex.  In  general  terms, 
however,  the  same  three  communities  are  represented,  although  other  communities  are 
present  in  the  data  set.  There  is  approximately  a  65  percent  overlap  between  individuals 
who  completed  questionnaires  in  the  1982  wave  and  in  the  1986  wave  (the  questionnaires 
may  vary,  us  explained  in  the  next  section  of  this  report).  We  shall  refer  to  the  subsample 
of  individuals  who  completed  questionnaires  in  both  1982  and  1986  as  the  "lagged 
sample."  The  term  "lagged"  denotes  that  questionnaire  data  are  available  for  each  of  the 
two  waves,  but  it  does  not  suggest  that  the  scales  of  measurement  are  the  same,  as 
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implied  by  the  term  "repeated  measures."  Some  of  the  lagged  data  do  indeed  involve 
repeated  measures,  either  because  the  same  or  similar  questionnaire  was  employed  in  both 
waves  or  because  scales  from  different  questionnaires  are  the  same,  or  with  scale 
transformations,  can  be  made  the  same.  Other  lagged  data  do  not  take  the  form  of 
repeated  measures.  An  example  is  items  and  scales  that  are  not  common  to  the  Aviation 
Officer  Career  Questionnaire  administered  in  the  1982  wave  and  the  Retirement  from 
Navy  Life  Questionnaire  administered  in  the  1986  wave. 

The  lagged  portion  of  the  data  set  extends  the  potential  career  span  represented  in 
the  data  to  as  much  as  25  years.  It  also  provides  the  opportunity  to  conduct  longitudinal 
analyses  such  as  cross-sectional  time  series  with  repeated  measures  data,  longitudinal 
prediction  with  lagged  data  (but  nonrepeated  measures),  and  quasi-experimentation.  The 
basic  data  design  underlying  such  analyses  is  a  cohort  by  years  commissioned  matrix  with 
lagged  data,  such  as  provided  by  Figure  4  of  Morrison  and  Cook  (1985). 

Nonlagged  data  are  also  represented  in  the  1986  wave.  Included  here  are  data  from 
officers  commissioned  after  the  1982  wave  as  well  as  officers  commissioned  in  the  1961 
to  1980  time  span  who  did  not  complete  questionnaires  in  the  1982  wave.  The  former 
subsample  adds  five  cohorts  to  the  analytic  design.  Members  of  the  nonlagged  1986  wave 
may  be  employed  in  appropriate  cross-sectional  analyses  of  data  from  the  1986  wave  and 
in  tests  of  the  reactivity  of  the  questionnaire  procedures. 

There  is  also  a  set  of  nonlagged  1982  wave  data.  Technically,  nonlagged  in  this 
context  refers  to  individuals  who  failed  to  complete  any  form  of  questionnaire  in  the  1986 
wave.  These  individuals  may  be  employed  in  cross-sectional  analyses  on  the  1982  data  set 
and  again  to  check  reactivity  to  questionnaire  procedures.  They  might  also  be  employed 
in  some  forms  of  lagged  analysis  if  they  possess  nonquestionnaire  data  on  longitudinal 
criteria  such  as  resignation  or  retirement. 

In  sum,  a  rather  complex  sampling  design  was  implemented.  Figure  1  presents  a 
summary  description  of  the  potential  samples  for  one  community.  The  symbol  "Xi"  refers 
to  questionnaire  data  collected  in  the  1982  wave;  "X2"  refers  to  questionnaire  data— from 
any  questionnaire— collected  in  the  1986  wave.  The  symbol  Yi  refers  to  any  criterion 
(questionnaire-based  such  as  intent  to  resign/retire  or  nonquestionnaire-based  such  as 
outcome  variables  from  the  Officer  Master  File)  collected  in  the  1982  wave,  whereas  Yj 
refers  to  any  criterion  collected  in  the  1986  wave.  Lagged  samples  are  indicated  by  (1) 
the  presence  of  X\  Yj  data  and  X2Y2  data,  and  (2)  a  solid  line  between  Xj  Yj  and  X2Y2. 
Nonlagged  samples  with  1982  wave  questionnaire  data  (but  not  1986  wave  questionnaire 

data)  are  represented  by  XjYj - Y2;  the  dashed  line  denotes  that  longitudinal  analyses 

may  be  conducted  only  for  Y2  criterion  data  collected  by  means  other  than  a  question¬ 
naire.  Nonlagged  samples  with  only  1986  wave  data  have  only  an  X2Y2  representation. 
Note  that  the  five  cohorts  added  in  the  1986  wave  questionnaire  administration  (Cohorts  1 
through  5  at  the  bottom  of  Figure  1)  have  only  X2Y2  data. 

An  issue  addressed  later  in  this  report  is  whether  sample  size  should  be  maximized 
for  any  particular  analysis  versus  whether  comparability  in  sample  size  should  be 
maintained  over  different  analyses.  We  also  consider  questions  such  as  how  cohorts  may 
be  combined  and  the  use  of  cross-sectional  versus  longitudinal  analytic  designs. 

Instruments 

A  truly  massive  amount  of  information  has  been  collected  in  this  project.  A  key 
source  of  information  is  provided  by  various  forms  of  the  Officer  Career  Questionnaire 
(OCQ),  each  of  which  contains  approximately  1,000  items.  Three  versions  of  the  OCQ 


were  administered  in  the  1982  wave;  the  different  forms  correspond  to  the  three 
communities  and  reflect  community-level  differences  in  organizational  policy  and  career 
paths  (Morrison  <5c  Cook,  1985).  However,  the  three  forms  share  a  common  core  of  688 
items. 

Somewhat  revised  forms  of  the  three  versions  of  the  OCQ  were  administered  in  the 
1986  wave.  In  addition,  separate  questionnaires  were  developed  for  individuals  who  had 
retired  from  the  Navy  (n  =  700,  80%  of  whom  have  1982  wave  data),  resigned  from  the 
Navy  (2  questionnaires,  n  =  1,250),  or  changed  designators  (e.g.,  changed  from  unrestricted 


3 


line  to  restricted  line,  or  changed  from  one  unrestricted  line  community  to  another 
unrestricted  line  community). 

A  second  basic  source  of  data  is  the  Officer  Master  File  (cf.  Morrison,  Martinez,  <5t 
Townsend,  19S4).  This  file  contains  historical  data  on  Navy  officers  such  as  year  of 
commissioning,  commissioning  source,  educational  background,  promotion  dates,  and 
acquisition  of  nonwarfare  skills.  The  Officer  Master  File,  in  combination  with  the  Officer 
Attrition  Tape,  is  also  the  source  for  behavioral  data  on  attrition  (cf.  Bruce  &  Burch,  in 
review). 

The  third  and  final  source  of  data  considered  here  consists  of  changes  in  organi¬ 
zational  policies  and/or  sociocultural  factors  that  influence  career  decisions  and  develop¬ 
ment  (Morrison  &  Cook,  1985).  A  straightforward  measurement  of  reactions  to  policy 
change  is  reflected  in  the  1986  questionnaire  for  URL(G)s  in  regard  to  detailing  (see 
question  20,  page  S).  However,  in  general,  policy  changes  and  changes  in  the  sociocultural 
environment  that  may  influence  career  decisions  are  being  tracked  by  the  researchers 
(e.g.,  changes  in  the  labor  market).  Such  information  may  be  employed  to  explain  results 
of  lagged  forms  of  analyses,  such  as  changes  in  measurement  models  (i.e.,  factor 
structures)  in  the  1982  wave  versus  the  1986  wave.  Moreover,  if  changes  are  quantifiable 
and  vary  for  different  groups  of  respondents  (e.g.,  a  new  policy  had  been  implemented  in 
some  geographic  locations  but  not  in  others  at  the  time  of  1986  wave  measurement),  then 
some  form  of  quasi-experimental  analysis  with  nonequivalent  control  groups  may  be 
implemented. 

We  shall  not  attempt  to  explic;  te  all  the  variables  contained  in  these  various  sources 
of  data.  A  basic  overview  and  brief  description  of  variables  is  provided  by  Morrison  and 
Cook  (1985,  pages  22-24),  who  also  furnish  overviews  of  the  development  of  the  Officer 
Quality  Index  (see  Morrison  et  al.,  1984  for  a  more  detailed  description  of  this  study)  and 
measures  of  career  satisfaction.  Examination  of  the  questionnaires  indicates  the  breadth 
of  the  domains  of  measurement.  Recent  reports  by  Bruce  and  Burch  (in  review)  and 
Wilcove,  Bruni,  and  Morrison  (1987)  illustrate  not  only  modeling  to  address  specific 
research  questions  with  selected  variables,  but  also  the  overall  quality  of  the  data.  The 
report  by  Morrison  (19S3)  on  interviews  with  SWOs  and  the  series  of  reports  on  studies  of 
Surface  Warfare  Junior  Office  Retention  furnish  valuable  background  information. 

MODELS 

Theoretical  Models  of  Career  Development 

Theoretical  models  of  career  development  in  the  Navy  operate  at  two  levels  of 
specificity.  The  less  specific,  more  global  level  of  explanation  is  presented  by  Morrison 
and  Cook  (1985).  Based  on  "a  comprehensive  review  of  career  stage,  decision  making,  and 
developmental  theories”  (p.  v),  Morrison  and  Cook  proposed  a  general  model  for  career 
development.  Salient  propositions  and  concerns  included  in  this  general  model  are  as 
follows: 

1.  Career  perceptions,  decisions,  and  behavior  are  a  function  of  interactions 
between  individual  characteristics  (e.g.,  needs,  values,  abilities),  organizational  factors 
(e.g.,  structure,  policies,  requirements,  communication  processes),  social  factors  (e.g., 
family  support,  social  influences  from  peers),  and  both  actual  and  perceived  influences 
engendered  by  sociocultural  environmental  factors,  including  changes  in  these  factors 
(e.g.,  economy,  labor  market,  political  environment).  These  propositions  are  reflective  of 


an  overall  person-environment  fit  model  (Endler  <3c  Magnusson,  1976;  Pervin,  1968;  Van 
Harrison,  197S). 

2.  Causal  models  linking  the  factors  above  the  career  perceptions,  decisions,  and 
behaviors  are  dynamic,  which  is  to  say  subject  to  change  over  time.  Change  may  be 
reflected  in  many  ways,  two  examples  being  changes  in  Navy  policy,  which  function  as 
main  effects  for  career  variables  as  well  as  moderators  of  causal  relations  between  career 
variables  and  other  causal  factors,  and  changes  in  the  values  and  attitudes  of  different 
cohorts  in  the  research  samples  (e.g.,  generational  differences),  which  may  also  function 
as  both  main  effects  and  moderators.  As  noted  by  Morrison  and  Cook  (1985),  to  be 
successful,  "organizations  should  anticipate  and  adapt  to  changing  values,  attitudes,  and 
priorities  of  current  and  potential  employees"  (p.  1). 

3.  Adherence  to,  versus  deviation  from,  structured  career  paths  receives  major 
consideration  in  this  model.  Operationalized  in  terms  of  the  research,  issues  of 
importance  pertain  to  whether  individuals  remain  in  unrestricted  line  positions  exclu¬ 
sively,  remain  in  unrestricted  line  positions  but  achieve  skills  in  a  subspeciality  through 
training  courses  or  formal  graduate  education,  or  transfer  to  a  new  occupational 
speciality  in  the  restricted  line  or  staff. 

4.  The  career  paths  for  individuals  are  themselves  dynamic.  The  logic  here  is  based 
on  life-span  development  theory,  which  suggest  that  career  decisions  involve  a  series  of 
major  and  minor  choices  over  time.  While  it  is  expected  that  prior  choices  and  decisions 
will  influence  later  choices  and  decisions,  it  is  also  possible,  indeed  likely,  that  the  causal 
factors  of  salience  in  early  career  decisions  are  different  from  the  causal  factors  of 
salience  in  middle  and  late  career  decisions.  In  other  words,  causal  factors  for  career 
decisions  are  sequentially  moderated  by  time,  which  reflects  variation  in  factors  such  as 
age,  maturation,  and  changing  value  systems  (James,  Joe,  &  Irons,  1982;  James  &  Tetrick, 
1984).  It  is  also  noteworthy  that  characterizing  career  growth/development/maturation  in 
terms  of  a  dynamic,  sequential  moderation  model  ties  directly  into  the  current  contro¬ 
versy  on  cross-situational  consistency  versus  cross-situational  specificity  of  behavior  (cf. 
Epstein,  1979,  1980,  1983;  Epstein  &  O'Brien,  1985;  James  <5 c  Deviney,  1988;  Mischel  & 
Peake,  1982;  Pervin,  1985).  In  effect,  the  Morrison  and  Cook  model  takes  the  fo  m  that  is 
known  as  "coherence"  model,  which  states  that  while  behavior  is  cross-situationally 
specific  in  a  relative  sense  (cf.  Endler  <5c  Magnusson,  1976),  the  dynamic  profiles  of 
specific  groups  of  individuals,  reflecting  behaviors  over  time  and  in  different  environ¬ 
ments,  are  predictable.  This  is  a  rather  different  model  than  the  "trait"  or  cross- 
situational  consistency  model  prevalent  in  selection  and  classification  research,  and 
represents  a  major  step  forward  in  personnel  research. 

5.  Yet  another  aspect  of  the  proposed  dynamic  model  is  the  cyclical  nature  of 
relations  among  the  causa!  factors.  An  example  of  a  cyclic  model  is  the  now  classic 
psychological  success  paradigm  presented  by  Hall  (1971)  and  reported  in  Figure  3  by 
Morrison  and  Cook  (1985).  Key  predictions  that  evolve  from  this  model  are  (a)  persons 
and  situations  reciprocally  interact  with  one  another  (cf.  Endler  <3c  Magnusson,  1976)  and 
thus  the  person-environment  fit  model  should  be  extended  to  include  the  effects  of 
individuals  on  situations  (e.g.,  the  types  of  individuals  who  self-select  and/or  are  formally 
selected  for  a  particular  environment  influence  the  characteristics  of  that  environment), 
and  (b)  perceptions  of  self-efficacy  (e.g.,  self-confidence,  self-esteem),  affect  (e.g.,  job 
satisfaction),  motivation,  and  behavior  both  engender  and  are  engendered  by  salient 
perceptual  factors  (e.g.,  challenge,  support,  autonomy),  and  thus  changes  in  any  factor 
(e.g.,  challenge)  may  permeate  the  entire  (open)  system  over  time. 
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While  not  exhaustive,  these  appear  to  be  among  the  main  facets  of  the  Morrison  and 
Cook  genera!  model  of  career  development  and  decision  making.  From  this  general  model 
are  derived  a  number  of  research  hypotheses,  which  brings  us  to  the  second  level  of 
specificity.  Morrison  and  Cook  (1985)  present  a  number  of  examples  of  research 
hypotheses  (pp.  6-10)  as  well  as  methodological  concerns  that  should  be  considered  in 
attempts  to  test  research  hypotheses  (pp.  12-17).  Moreover,  as  reviewed  earlier,  the 
Bruce  and  Burch  (in  review)  and  Wilcove  et  al.  (1987)  reports  demonstrate  initial  attempts 
to  test  specific  research  hypotheses  that  evolved  from  the  general  model. 

Based  on  discussions  with  members  of  the  PDCD  work  unit,  two  specific  models  are 
of  prime  interest  at  this  time.  The  first  model  will  be  designed  to  explain  the  reasons  why 
officers  stay  in  the  Navy  versus  leave,  and  is  relerred  to  as  the  "retention  model."  The 
second  model  will  attempt  to  explain  why  officers  stay  operational  (i.e.,  remain  in 
unrestricted  line  positions)  versus  (1)  remain  operational  but  obtain  a  subspeciality  or  (2) 
transfer  to  a  specialized  occupation  in  restricted  line  or  staff  (i.e.,  change  designators). 
From  an  analytic  standpoint,  the  two  (sets  of)  models  are  similar  and  thus,  for  illustrative 
purposes,  we  consider  only  the  retention  model  in  this  report. 

It  is  unlikely  that  a  single  retention  model  will  suffice  to  explain  all  retention 
decisions.  Rather,  it  is  probable  that  a  set  of  retention  models  will  need  to  be  developed 
and  tested  empirically.  The  need  for  a  set  of  retention  models  derives  from  the  likelihood 
that  a  series  of  boundary  conditions  exist  in  the  overall  data  set,  which  is  to  say  that  a 
series  of  moderators  exist  that  bound  or  limit  the  generalizability  of  a  particular  model  to 
only  a  subset  of  the  overall  sample.  In  other  words,  different  models  will  need  to  be 
developed  for  different  subgroups  of  officers. 

Two  examples  of  potential  moderators  are  career  stage  and  community.  To  illustrate 
the  role  played  by  the  first  moderator,  suppose  analyses  are  conducted  separately  for 
communities,  with  SWOs  as  an  initial  choice.  The  general  model  advanced  by  Morrison 
and  Cook  (1985)  predicts  that  the  causal  models  for  retention  in  the  SWO  community  are 
sequentially  moderated  by  career  stage.  Morrison  (1983)  suggests  further  that  career 
choice  points  "appear  to  revolve  around  assignment  decisions"  (p.  A-2),  and  thus  one  might 
use  career  choice  points  as  the  basis  for  grouping  cohorts  into  sequentially  moderated 
subgroups  (not  all  cohorts  may  be  included  in  this  analysis).  For  example,  based  on 
Morrison  (1983),  the  following  four  SWO  subgroups  might  be  developed:  (1)  officers  who 
have  been  in  the  Navy  for  18  to  30  months,  (2)  officers  who  have  just  completed  a 
department  head  tour,  (3)  officers  who  have  recently  been  screened  for  executive  officer, 
and  (4)  officers  who  have  recently  been  screened  for  commanding  officer.  These  four 
subgroups  form  "social  cohorts"  (Morrison  &  Cook,  1985,  p.  15)  in  the  sense  that  members 
of  each  social  cohort  have  recently  shared  a  life  experience  (although  the  results  of  the 
experience  are  not  the  same  for  all  members  of  a  cohort),  the  experience  is  structured, 
the  experience  has  a  common  generational  framework  that  distinguishes  one  generation 
from  another,  and  the  experience  is  likely  to  have  long-term  effects. 

A  model  of  retention  may  be  developed  for  each  of  the  four  sequentially  moderated 
subgroups  or  social  cohorts.  An  illustrative  model  tested  by  Bruce  and  Burch  (in  review) 
for  Navy  aviators  is  stm  ^  in  pigure  2.  This  is  a  linkage,  sequencing,  or  mediation 
variable  model  (James  ’■  ‘^rett,  1984),  or  in  statistical  terms,  a  recursive  model  with 
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Figure  2.  Retention  model  for  naval  aviators. 


longitudinal  retention  data  (the  remainder  of  the  model  is  cross-sectional).  For 
illustrative  purposes,  we  shall  represent  the  model  as: 


where  the  x's  refer  to  cross-sectional  data  (from  questionnaires  and  the  Officer  Master 
File)  and  Yj  refers  to  longitudinal  retention  data.  Also  for  illustrative  purposes,  we  shall 
assume  that  the  x's  and  y's  are  the  same  for  all  social  cohorts,  realizing  that  this  may  not 
be  the  case  in  the  final  explanatory  models  developed  for  a  specific  cohort.  The  same 
variable  sets  allow  for  more  direct  tests  of  the  statistical  similarity  of  the  models  over 
cohorts,  which  are  based  on  homogeneity  of  regression  tests  (cf.  Johnston,  1984). 
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A  set  of  potential  analytic  models  is  represented  in  Figures  3  through  5.  Figure  3 
presents  eight  replications  of  the  model;  one  replication  for  each  of  four  cohorts  in  each 
of  two  waves  of  measurement  (y  is  again  measured  longitudinally  in  each  replication). 
Within  each  wave  of  measurement,  the  four  models  representing  different  cohorts  may  be 
contrasted  simultaneously  by  estimating  the  structural  coefficients  for  each  model 
separately,  and  then  using  linear  contrasts  to  ascertain  if  the  structural  coefficients  for 
the  four  models  are  the  same  or  different  (James  dc  James,  19SS;  Johnston,  1984).  More 
specific  contrasts  may  be  made  between  one  or  more  subsets  of  models  and  theoretically 
viable  a  priori  hypotheses  may  be  tested  using  a  form  of  planned  comparisons  (James  & 
James,  1988).  An  alternative  and  quicker  method  of  analysis  is  to  employ  LISREL  to 
conduct  the  above  homogeneity  of  regression  analyses.  In  the  LISREL  analyses,  all 
parameters  for  the  four  cohort  models  would  be  estimated  simultaneously  (under  an 
equality  constraint)  and  then  a  form  of  homogeneity  test  conducted. 
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Figure  3.  Retention  models  x  social  cohort  x  wave  measurement. 
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"■The  first  subscript  refers  to  variable  and  the  second  subscript 
refers  to  time  of  measurement  were  1  =  1982  and  2  =  1986. 

Figure  4.  Nonlagged  time  series  for  repeated  measures  data  with  a 
social  cohort. 
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Figure  5.  Lagged  endogenous  variable  time  series  within  a  social 
cohort. 
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The  analysis  within  each  wave  of  measurement  provides  information  as  to  whether 
causal  models  for  retention  vary  by  career  stage.  There  are,  however,  many  confounds  in 
the  data  (cf.  Morrison  &  Cook,  1985,  pp.  12-17).  In  large  part,  the  confounds  relate  to  the 
fact  that  sequential  moderation,  with  career  stage  as  the  moderator,  refers  to  a  cross- 
sectional  time-series  design  (i.e.,  a  form  of  repeated  measures)  where  the  same 
individuals  are  followed  over  career  stages.  The  analyses  described  above  reflect  an 
independent  groups  design  rather  than  a  repeated  measures  design,  where  differences 
among  diverse  groups  of  different  individuals  (at  different  career  stages)  are  used  to  infer 
what  would  happen  to  the  same  individuals  over  career  stages  (cf.  James  &  Singh,  1978). 
The  problem,  of  course,  is  that  results  based  on  independent  groups  in  a  cross-sectional 
design  may  not  be  perfectly  congruent  with  results  based  on  following  the  same 
individuals  over  career  stages.  This  problem  is  solved  by  transferring  to  a  time-series 
format  of  analysis.  First,  we  consider  another  form  of  analysis  based  on  Figure  3. 

This  analysis  is  predicated  on  the  possibility  that  results  of  the  cross-sectional, 
independent  groups  analyses  described  above  are  different  for  the  two  waves  of 
measurement.  Variation  of  results  as  a  function  of  wave  of  measurement  are  suggestive 
of  (1)  changes  in  Navy  policy  or  environmental  factors  during  the  period  intervening 
between  1982  and  1986  or  (2)  the  causal  models  for  the  same  decision  (e.g.,  whether  to 
remain  in  the  Navy  beyond  the  Minin, um  Service  Requirement)  but  for  different  cohorts 
(e.g.,  officers  in  the  18  to  30  month  stage  in  the  1982  wave  versus  officers  in  the  18  to  30 
month  stage  in  the  1986  wave)  are  themselves  different  (i.e.,  a  "longitudinally"  defined 
cohort  difference).  Explanations  "1"  and  "2"  may  both  be  operative  as  well,  which 
indicates  the  need  for  documentation  of  changes  in  Navy  policy  and  the  sociocultural 
environment  and,  if  possible,  quasi-experiental  analyses  to  investigate  the  effects  of  such 
changes  on  the  data. 

In  the  present  context,  homogeneity  of  regression  analyses  are  indicated  wherein 
models  for  the  1982  wave  are  contrasted  with  models  for  the  1986  wave.  Such  analyses 
are  not  a  form  of  time  series  because  different  groups  of  individuals  are  being  contrasted 
(i.e.,  longitudinal  cohorts).  Analyses  may  be  conducted  for  one  or  more  particular  career 
stages  (e.g.,  the  1982  model  vs.  the  1986  model  for  the  two  18  to  30  month  cohorts), 
subsets  of  career  stages,  or  all  career  stages  simultaneously.  Analytic  methods  would  be 
the  same  as  described  above. 

We  now  proceed  to  time-series  forms  of  analyses,  which  technically  are  referred  to 
as  cross-sectional  time  series  inasmuch  as  measurements  are  taken  on  multiple  (but  the 
same)  individuals  at  two  or  more  points  in  time  (cf.  Nerlove,  1971;  Hannan  It  Young,  1977; 
Johnston,  1984).  (Time  series  usually  refers  to  single  subject  designs.)  The  first  form  of 
time-series  designed  is  a  nonlagged  model,  as  shown  in  Figure  4.  The  defining 
characteristics  of  a  nonlagged  time  series  are  (1)  all  causal  effects  take  place  with  a  time 
interval  (e.g.,  the  time  interval  encompassed  by  the  1982  wave  of  measurement)  and  (2) 
there  are  NO  lagged  effects  from  one  time  interval  to  the  next  time  interval  (e.g.,  Career 
Intent  in  1982  has  no  effect  on  Career  Intent  in  1986). 

Figure  4  presents  a  nonlagged  time  series  for  explaining  Career  Intent  for  a  selected 
social  cohort.  Since  the  same  officers  are  involved  in  the  two  time  intervals  (i.e., 
repeated  measures),  the  measurements  on  variables  subject  to  change  (e.g.,  Career  Intent, 
organizational  commitment)  reflect  perceptions,  attitudes,  and  values  for  different  career 
stages.  According  to  the  general  model  of  Morrison  and  Cook  (1985),  we  would  expect  the 
structural  parameters  in  the  two  models  to  vary.  That  is,  comparison  of  the  1982  model 
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with  the  19S6  model  via  homogeneity  of  regression  tests  that  take  correlated  errors  and 
structural  coefficients  into  account  (James  5c  Tetrick,  1984)  would  be  expected  to 
indicate  moderation  by  time  (career  state).  Such  moderation  is  interpreted  to  mean  that 
the  models  are  nonstationary. 

It  is  noteworthy  that  if  lack  of  moderation,  or  stationarity,  were  indicated,  then  new 
methods  being  developed  by  James  and  colleagues  could  be  employed  to  correct  for 
unmeasured  variables  problems.  Unmeasured  variables  problems  (or  lack  of  self¬ 
containment)  are  the  key  problem  with  the  use  of  confirmatory  analytic  designs  on  field 
data  (James,  1980;  James,  Mulaik,  5c  Brett,  1982).  The  techniques  being  developed  by 
James  and  colleagues  use  an  estimate  of  the  autocorrelation  between  disturbances  (see 
Figure  4)  and  generalized  least  squares  to  effect  a  statistical  correction  for  this  problem. 
We  will  address  this  issue  further  in  our  second  report,  but  for  now  note  that  this  process 
is  likely  nonapplicable  in  the  context  of  the  hypotheses  generated  by  the  Morrison  and 
Cook  model. 

The  final  illustrative  model  presented  in  this  report  is  a  lagged  time  series  (see 
Figure  5).  Technically,  this  is  not  a  true  time-series  model  because  a  third  wave  of 
repeated  measures  is  needed  to  test  for  stationarity.  Nevertheless,  the  design  takes  the 
form  of  a  lagged  time  series  wherein  variables  measured  at  one  point  in  time  (1982)  are 
viewed  as  causes  of  variables  measured  at  a  different  point  in  time  (1986).  The  model  is  a 
form  of  lagged  endogenous  variable  time  series  (James  5c  Singh,  1978)  because  a  key 
endogenous  variable-Career  Intent~is  viewed  as  a  cause  of  itself  (i.e.,  the  causal  arrow 
from  X^l  to  X^2)- 

Like  Figure  4,  Figure  5  involves  repeated  measures  on  a  single  cohort,  only  here  the 
objective  is  not  so  much  to  compare  two  sets  of  structure  coefficients  as  it  is  to  estimate 
structural  coefficients  for  the  1986  measure  of  Career  Intent  (X^)  and  the  longitudinally 
measured  retention  variable  (Y12).  Many  arrows  could  be  added  or  removed  from  this 
illustrative  model— these  decisions  depend  on  theory  and  the  cohort  being  analyzed.  The 
important  issue  here  is  that  this  "quasi-lagged"  form  of  time  series  corresponds  more 
closely  to  the  hypotheses  advanced  by  Morrison  and  Cook  (1985)  than  the  nonlagged  model 
in  Figure  4.  However,  without  a  third  wave  of  data,  the  techniques  being  developed  by 
James  et  al.  (1982)  cannot  be  used  to  effect  statistical  controls  for  unmeasured  variables. 
Consequently,  it  will  be  important  to  apply  the  decision  rules  advocated  by  James  (1980) 
to  decide  whether  analyses  should  focus  on  prediction  without  causal  inferences  versus 
causal  analysis.  Numerous  analytic  techniques  are  available,  including  basic  cross- 
sectional  time  series,  generalized  least  squares,  and  LISREL.  We  will  address  this  issue 
specifically  in  our  second  report. 


SPECIFIC  ANALYTIC  AND  STATISTICAL  CONCERNS 

This  report  is  concluded  with  brief  overviews  of  specific  analytic  and  statistical 
issues.  Please  note  that  our  second  report  is  to  be  devoted  to  analyses  and  statistics, 
where  we  will  consider  a  broader  selection  of  models,  including  quasi-experimental 
models,  assumptions  for  analyses,  and  specific  analytic  techniques.  At  this  time  we 
consider  statistical  issues  that,  while  receiving  greater  attention  in  the  second  report,  the 
investigators  may  wish  to  consider  in  preparing  for  analyses. 

1.  Confirmatory  analytic  methods  (path  analysis,  structural  equation  analysis, 
LISREL)  designed  to  estimate  causal  parameters  will  be  used  frequently.  While  these 
analyses  will  be  guided  by  the  general  career  development  model  and  models  specific  to 
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selected  endogenous  variables,  it  is  likely  that  initial  causal  models  will  be  disconfirmed. 
Hence,  like  the  Bruce  and  Burch  (in  review)  analysis,  it  will  be  necessary  to  conduct 
"exploratory"  forms  of  analysis  to  build  data-based  causal  models.  Exploratory  forms  of 
analysis  designed  to  develop  causal  models  with  good  statistical  fits  to  data  are  referred 
to  as  "specification  analyses"  (cf.  MacCallum,  1986).  Models  developed  by  specification 
analyses  need  to  be  tested  with  new  data — a  form  of  cross-validation  (Cudeck  &  Browne, 
19S3).  Hence,  we  recommend  the  use  of  cross-validation  or  hold-out  samples  whenever 
possible. 

2.  The  investigators  have  a  choice  of  using  latent  variable  versus  manifest  variable 
designs  in  the  confirmatory  analyses,  or  perhaps  some  mixture  of  the  two  (cf.  James  et 
al.,  1982).  When  possible,  we  recommend  the  use  of  latent  variable  designs.  In  part,  this 
recommendation  is  based  on  the  points  in  favor  of  latent  variable  designs  reviewed  in 
James  et  al.  (1982— e.g.,  use  of  perfectly  reliable  variables  in  structural  portions  o' 
models).  This  recommendation  is  also  based  on  issues  of  special  importance  to  this 
particular  data  set,  such  as  the  following: 

a.  A  number  of  item  composites  or  scales  now  being  used  as  manifest  variables 
are  based  on  only  a  few  items,  with  accompanying  modest  (although  often  acceptable) 
coefficient  alphas.  This  is  likely  to  raise  a  red  flag  in  future  scientific  reviews  and 
journal  reviews  of  reports  and  manuscripts.  Where  possible,  we  suggest  that  subsets  of 
the  present  manifest  scales  be  treated  as  indicators  of  higher-order,  more  explanatory 
latent  variables,  which  in  turn  could  be  used  in  the  structural  portions  of  confirmatory 
analyses.  Note  that  the  latent  variables  could  be  developed  in  part  by  exploratory 
analysis  (inspection  of  correlations  of  manifest  scales,  higher-order  exploratory  factor 
analysis)  if  the  cross-validation  effort  recommended  in  "1"  above  is  adopted  (Schmitt, 
1987).  (One  would  then  be  cross-validating  measurement  models  as  well  as  structural 
models.) 


b.  The  repeated  measures  portions  of  lagged  designs  (e.g.,  Figure  5)  involve 
readministration  of  the  same  questionnaire  items  and  scales.  The  likelihood  exists  that 
measurement  errors  for  manifest  indicator  variables  (i.e.,  the  scales  or  item  composites) 
will  be  autocorrelated  over  time.  Within  the  constraints  of  identification,  correlated 
measurement  errors  may  be  included,  and  tested,  in  the  measurement  sections  of  latent 
/ariable  models. 

c.  The  use  of  a  comparatively  large  number  of  manifest  causal  variables  in 
confirmatory  analyses  increases  the  probability  of  multicollinearity  (cf.  Gordon,  1968). 
An  associated  problem  encountered  in  latent  forms  of  analysis  is  "empirical  under 
identification,"  which  often  occurs  when  two  or  more  first-order  factors  are  highly 
correlated  and  hence  are  possibly  indicators  of  the  same  underlying  latent  construct 
(Dillon,  Kumar,  Sc  Mulani,  1987).  A  potential  solution  to  both  problems  is  to  identify 
higher-order  latent  variables  into  which  highly  correlated  manifest  variables/first-order 
factors  can  be  clustered. 

d.  Morrison  and  Cook  (1985)  note  the  need  to  compare  factor  structures  for 
different  cohorts  as  well  as  for  repeated  measurements  on  the  same  cohorts.  Tests  of  the 
measurement  portions  of  latent  variable  models  via  LISREL  address  this  analytic  need 
directly.  Moreover,  additional  measurement  models  for  nonlagged  data  (i.e.,  data  from 
officers  who  participated  in  only  the  1982  wave  or  the  1986  wave)  may  be  developed  and 
contrasted  with  measurement  models  for  repeated  measures  data.  Such  contrasts  allow 
for  partial  tests  of  reactivity  resulting  from  completing  questionnaires  on  multiple 
occasions. 
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3.  It  is  suggested  that  calculation  and  use  of  change  scores  (of  any  form)  be  avoided 
(Cronbach  &  Furby,  1970).  In  place  of  change  score  analysis  in  quasi-experimental 
designs,  we  shall  recommend  the  use  of  hierarchical  regression  techniques  (which  are 
more  flexible  than  covariance  designs,  see  Cohen  &  Cohen,  1983). 

4.  How  to  cluster  cohorts  is  a  primary  question  raised  by  Morrison  and  Cook  (1985). 
Any  attempt  to  cluster  the  present  cohorts  defined  by  year  of  commissioning  will  require 
tests  to  ascertain  whether  descriptive  data  are  similar  versus  different  (e.g.,  distribu¬ 
tional  data  such  as  means  and  standard  deviations),  whether  psychometric  data  are  similar 
versus  different  (e.g.,  reliabilities,  measurement  models),  and  whether  structural  coeffi¬ 
cients  are  similar  versus  different  (i.e.,  homogeneity  of  regression  tests).  We  suggest  that 
knowledge  of  Navy  practices  and  policy  combined  with  predictions  generated  by  the 
career  development  model  or  models  specific  to  endogenous  variables,  such  as  retention, 
be  employed  to  define  clusterings  of  cohorts.  An  example  is  the  clustering  by  similarity 
of  career  stage  and  decision  points  illustrated  in  Figure  3.  Such  clusters  have  a 
substantive  base  and  may  be  checked  empirically.  A  less  desirable  method  is  to  cluster 
simply  on  the  basis  of  empirical  similarity  using  profile  analytic  techniques.  A  key 
problem  with  these  techniques  is  that  25  percent  or  more  of  the  sample  usually  fail  to  fit 
unambiguously  into  a  cluster,  which  stimulates  generalizability  problems. 

5.  Moderator  or  homogeneity  of  regression  tests  represent  a  central  theme  through¬ 
out  the  analyses.  It  is  generally  recommended  that  such  tests,  as  well  as  any  form  of  time 
series  analysis,  be  predicated  on  nonstandardized  data  (cf.  James  et  al.,  1982).  This  raises 
the  problem  that  some  item  scales  were  changed  in  the  1986  wave  questionnaires  in 
relation  to  the  1982  wave  questionnaires.  We  recommend  that  an  attempt  be  made  to 
rescale  the  "longer"  item  scales  to  make  them  comparable  to  the  "shorter"  item  scales 
and  thereby  preserve  a  form  "raw  scores."  Rescaling  should  be  predicated  comparisons  of 
distributions  and  correlations,  and  employ  both  lagged  as  well  as  noniagged  data  (the 
latter  data  allow  for  checks  on  reactivity  such  as  carry-over  effects  and  both  beta  and 
gamma  change— see  Golembiewski,  Billingsley,  <Jc  Yeager,  1976). 


6.  The  availability  of  both  lagged  and  noniagged  questionnaire  data  introduces  the 
interesting  question  of  whether  to  attempt  to  maximize  sample  sizes  in  cross-sectional 
studies  by  including  officers  who  completed  questionnaires  in  both  waves  of  measurement 
and  officers  who  completed  questionnaires  in  only  one  wave  (e.g.,  a  cross-sectional  study 
on  the  1982  wave  data  that  includes  both  officers  with  1986  wave  data  and  officers  with 
only  1982  wave  data).  While  such  an  approach  maximizes  power  and  generalizability,  it 
limits  attempts  to  pursue  additional  analyses  involving  lagged  and  repeated  measures  data 
on  a  comparable  sample.  When  both  cross-sectional  and  lagged  analyses  are  to  be 
conducted,  we  suggest  that  the  cross-sectional  analyses  be  conducted  on  two  samples, 
namely  (a)  officers  who  have  both  lagged  and  noniagged  data,  and  (b)  officers  who  have 
only  lagged  data.  Hopefully,  results  of  cross-sectional  analyses  will  be  similar  for  these 
two  samples.  If  this  is  not  the  case,  then  the  investigators  will  know  where  the  statistical 
differences  occur  and  may  take  these  differences  into  consideration  when  interpreting 
results  of  later  lagged  analyses. 

7.  A  similar  recommendation  applies  to  studies  on  "selected  samples."  An  example 
of  a  selected  sample  analysis  is  the  Bruce  and  Burch  (in  review)  study,  where  equal  sample 
sizes  of  (married)  stayers  and  leavers  were  selected.  Equal  sample  sizes  for  stayers  and 
leavers  is  a  solution  to  the  base-rate  problem,  especially  given  a  dichotomous  criterion 
and  the  use  of  point-biserial  correlation  (cf.  Lord  <5c  Novick,  1968).  However,  a  problem 
arises  when  one  wishes  to  judge  the  effectiveness  of  the  equations  in  explaining/predicting 
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retention  in  an  unseiected  population  wherein  the  stayers  and  leavers  are  not  equally 
represented  (i.e.,  the  base  rate  is  not  50%).  Our  opinion  is  that  the  use  of  equal  sample 
sizes  is  a  reasonable  means  for  developing  a  statistical  explanatory/prediction  system. 
However,  the  effectiveness  of  this  system  should  be  checked  by  application  of  this  system 
to  a  general,  unselected  sample  in  the  same  sense  that  initial  validation  systems  are 
cross-validated. 
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